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In this paper we systematically study the double layer vortex lattice model, which is proposed 
to illustrate the interplay between the physics of a fast rotating Bose-Einstein condensate and the 
macroscopic quantum tunnelling. The phase diagram of the system is obtained. We find that 
under certain conditions the system will exhibit one novel phase transition, which is consequence 
of competition between inter-layer coherent hopping and inter-layer density-density interaction. In 
one phase the vortices in one layer coincide with those in the other layer. And in another phase two 
sets of vortex lattices are staggered, and as a result the quantum tunnelling between two layers is 
suppressed. To obtain the phase diagram we use two kinds of mean field theories which are quantum 
Hall mean field and Thomas- Fermi mean field. Two different criteria for the transition taking place 
are obtained respectively, which reveals some fundamental differences between these two mean field 
states. The sliding mode excitation is also discussed. 



I. INTRODUCTION 

In the recent years remarkable progress has been made 
in the field of ultracold quantum gas, among which the 
achievement of fast rotating boson gases and the real- 
ization of Mott insulator to superfiuid transition are two 
important ones. The physics of these two phenomena 
have attracted a lot of theoretical and experimental in- 
terests. 

It has been long predicted that there exists a quantum 
phase transition from the superfiuid phase to the Mott 
insulator phase in the boson Hubbard model as the hop- 
ping parameter decreases, and recently it has been ob- 
served in ultracold Bose atoms in the optical lattice. [J 
The Mott insulator phase is characterized by the loss of 
phase coherence among the sites and the whole system 
becoming fragmented. The essential physics of this tran- 
sition can be demonstrated by a simpler model, namely 
the bosons confined in a double well potential. In the 
strong tunnelling limit, the condensates located in the 
two wells develop a relative phase and the ground state 
is described by a coherent state. In the opposite limit, 
the relative number fluctuation is suppressed and each 
condensate has determined atom number. The ground 
state is well described by a Fock state. Hence there exists 
a crossover from the coherent regime to the Fock regime 
as the parameter t/U decreasing in the two- well system. 

On the other hand, the boson atoms confined in the 
a quasi-two-dimensional harmonic trap can now be ro- 
tated fast enough that the rotation frequency is very 
close to trapping frequency. It has been observed 
that such rapid rotating condensate contains a large num- 
ber of vortices and they form a regular triangular lat- 
tice. The fast rotating BEC is also characterized by 
two regimes. One is called the Thomas-Fermi mean field 
(TFMF) regime, where the interaction energy dominates 
over the kinetic energy. The other is the quantum Hall 
mean field (QHMF) regime, in which the energy gap be- 
tween single particle Landau levels is much larger than 
the interaction energy. 5] Based on the quantum Hall 
mean field theory the two-component fast rotating BEC 



was firstly studied by Mueller and Ho.[^ The relative dis- 
placement ro between the two sets of vortex lattices was 
found to be a non-vanishing value for even a little pos- 
itive inter-special interaction Ui2. With the increase of 
Ui2 the vortex lattice will experience a structure transi- 
tion from triangular to square. Similar conclusions was 
later obtained by numerical study in the Thomas-Fermi 
regime, i?] All these have motivated us to present a novel 
model that contains the above two physical features and 
illustrates their interplay. 



II. THE MODEL 

In this section we introduce our model for double-layer 
rotating BEC. The BEC is confined in a harmonic trap in 
the xy plane and high angular momentum with respect to 
z direction is imparted into the condensate. Along the 
z direction the condensate is confined in a double well 
potential. Thus two sets of vortex lattices are formed 
in the wells and they couple to each other via quantum 
tunnelling through the barrier between the wells. We call 
such a system double-layer vortex lattice and schemati- 
cally illustrate it in FigQ] It is believed to be experimen- 
tally realizable due to the recent technique progress on 
double well potentialQQ- The conclusions drawn from 
this model can be directly generalized to the case that 
a fast rotating BEC is cut into pieces by applying an 
optical lattice along the z-direction as shown in FigQ] 

When these two layers are well separated by a high 
potential barrier, the wave function overlapping along 
the z-direction is sufficiently small, the phase fluctuation 
between the wells is strong enough that the inter-layer 
phase coherence is lost and the coherent hopping is sup- 
pressed. This regime is called Fock regime because the 
particle number in each well is almost fixed. From the 
work of Mueller and Ho 6] we know that in this regime 
the relative displacement rg will take a non-zero value 
due to the inter-layer density-density interaction. How- 
ever this effect may be frangible because this interaction 
is very weak in this case. 
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Here Vdou is the double well potential. ui^2 = (us±Wa)/2 
represent the wave packets localized in each well. Here 
Us and Ua are the normalized ground state and the first 
excited state of hz , whose eigenvalues are assumed to be 
far below other eigenvalues. Us and Ua are symmetric and 
antisymmetric respectively, the energy splitting between 
them is denoted by t. 

In the rotating frame, the single particle Hamiltonian 
in xy plane, h^.y, is 



FIG. 1: Left: The double-layer vortex lattice system; Right: 
The rotating BEG confined in a optical lattice in the z- 
direction. 
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While in the coherent regime the whole system is de- 
scribed by a macroscopic wave function 



* = \/2 [.Jrilui(z)Lpi{x,y) + ^/nie'-'^U2{z)ip2{x,v))] 



(1) 



and the energy functional in the rotating frame is written 
as 



H ~ J dxdydz 
where 



2to dz^ 



(2) 
(3) 



where uj± is the trapping frequency and Q is the rotation 
frequency. In the fast rotating condensate, f2 is quite 
close to LUj_ and the condensate in each well forms a vortex 
lattice, denoted by ipi and (f2 respectively. We define 
AiV = (ni —n2)N as the relative atom number difference 
between the two condensates, and 9 is the relative phase. 
Corresponding to a normalized ni+n2 should be equal 
to unity. 

Substituting the order parameterl^ into the energy 
functional Pj). the energy functional can be written as 
follows :|10| 



E = —t^nin2 ^cos^SR J dxdy(plip2 + sin 6"^ J dxdyLplf2^ + J dxdy [nitf\hxy^i -\- n2^P2hxy^2\ 
+ W (^l J dxdy\(fi\^ + nl J dxdy\(p2\*^ 



8Ui2nin2 ( 2 / dxdy\(pi\'^\(p2\'^ + cos29n 



dxdy{Lpl(p2)'^ ~ sin29^ J dxdy{ipl(p2) 



(5) 



5J and S stand for the real part and imaginary part re- 
spectively. Here we denote U = 2iTh'^ascN{J dzuf ^)/™ 
and Ui2 = 2TTh'^ascN{J dzu\u2)/m. In this case U is al- 
ways larger than U12 by at least one order of magnitude, 
so the energy minimum occurs at ni = n2 and the two 
layers have the same lattice type. 

The main goal of this paper is to perform a systematic 
study of the double-layer vortex lattice system based on 
the energy functionaljSl), including the mean field phase 
diagram, phase transition and excitations. Several key 
points will be contained. Let us first briefly introduce 
them in the following. 

As shown in the case of two-component rotating 
BEC 6] ^, the repulsive density-density interaction term 



|</3iPl'/52p favors a situation in which the high density re- 
gion of one condensate coincides with the low density re- 
gion of the other, therefore the vortices of the two conden- 
sates should avoid each other. The difference between the 
current model and the two-component case is the pres- 
ence of coherent hopping terms such as iplip2 and (fil^ifl, 
which favor the vortices in two layers being coincident. 
Hence the double-layer model is of interest because there 
exist competing terms in the energy functional, and it 
is natural to ask whether and how this competition will 
manifest itself in a quantum phase transition. 

We find out in this paper that such a transition does 
not always take place, and we will answer the question 
under which condition the competition will result in a 



phase transition. Because a qualitative investigation of 
the inter-layer physics directly relies on how we describe 
the vortex lattice state, and we know that there exist two 
different mean field regimes for the fast rotating BEC 
called Thomas-Fermi regime and quantum Hall regimes, 
which are distinguished by the radio of the kinetic energy 
to the interaction energy, we study the issue with two 
kinds of mean field theories respectively and obtain two 
different criteria for the inter-layer transition taking place 
in these two regimes. We will remark in the end of this 
paper that the difference reveals some intrinsic properties 
of these two mean field ansatzs. 

The model studied here is also different from the dou- 
ble well model because the condensate in each well has 
a vortex lattice structure of its own, which is beyond 
the single model approximation used in the discussion of 
double- well BEC. It is known that the transition from 
coherent regime to Fock regime is driven by the hopping 
element between two condensates. Here we will find that 
the hopping element, as well as relative phase fiuctuation, 
is not only dependent on the wave function overlapping 
along the z-direction as in the double well case, but also 
depend on the integral J dxdy(plip2- Therefore, when two 
sets of vortex lattices do not coincide with each other, 
the integral will be relatively small and the tunnelling 
between two layers will be suppressed. This presents a 
new mechanism for the transition from coherent regime 
to Fock regime, which is another key point discussed in 
this paper. 

For the case that the transition can occur, we analyt- 
ically give the phase boundary between the coincident 
lattice phase and the staggered lattice phase. For the 
case that the transition is absent, we find that the rela- 
tive phase between two condensates will experience a 
second order change ast/Ui2 decreases. After investigat- 
ing a new excitation mode called vortex lattice's sliding 
mode, we point out that such a change will manifest itself 
in the frequency of sliding mode. 

The paper is organized as follows. In the following sec- 
tion, we will first study the case that the vortex lattice 
state is in the QHMF regime. After a brief review of 
QHMF theory in the first subsection IIII Al we focus on 
the transition for triangular vortex lattice in the subsec- 
tion ^^B] and we discuss the sliding mode in the sub- 
section ^^Sl Then in the next subsection IIII Dl we will 
generalize our discussion to arbitrary lattice type and 
summarize the conclusions in the subsection llll El In the 
section llVl we focus on TFMF regime and present a wave 
function ansatz to describe vortex lattice state with the 
help of Thomas-Fermi approximation. Using this ansatz 
we will revisit the issue studied in the third section. The 
results obtained from the two different mean field theo- 
ries are compared. We remark in the last section that 
these difference distinguish the intrinsic properties of the 
two regimes. 
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FIG. 2: Left: The coincident lattice phase. Right: The stag- 
gered lattice phase. The main part of this paper are concerned 
with under which condition the transition between these two 
phases can take place and determining the phase boundary. 



III. QUANTUM HALL MEAN FIELD REGIME 

A. Review of QHMF Theory 

Here we first briefly review the quantum Hall mean 
field theory for single rotating condensate. We notice 
that hx^y can be rewritten as 

1 ^ 2 

hx y = i~ihV — 'mLO_\_z X r) + — ^)Lz ■ (6) 

The Hamiltonian is identical to that describing a two- 
dimensional particle moving in a perpendicular mag- 
netic field in the symmetric gauge. The eigenstates of 
hx,y are the Landau levels. Defining the complex vari- 
able w = (x + iy)/2aj_ and w* — {x ~ iy)/2a± with 
aj_ — \J h/jjiLo^^ in the quantum Hall mean field picture 
the system is forced into the lowest Landau level (LLL) 
under the condition that $7 is close to wj^ and Ml is much 
larger than the interaction energy, and consequently the 
macroscopic wave function (fi^2{x^ y) are analytical func- 
tions of the complex variable w apart from a Gaussian 
factor. In the LLL, the first term in Eq.® gives the same 
expectation value for all states and can be neglected from 
the energy functional. The expectation value of the single 
particle Hamiltonian turns out to be^ 

{hx.^y)^n{uj^-n){\w\^). (7) 

It is noticed that for the vortex lattice states all first 
order zeros of the entire functions (/?, which are the lo- 
cations of vortices, form a regular lattice. Besides, the 
condensate density should be circular symmetric if the 
confinement potential is isotropic. These two require- 
ments uniquely determine that ip should be in the fol- 
lowing form: 

^,h = 0(C,r)exp(^— jexp(^-^j. (8) 

Here d(C,T) is the Jacobi theta function. We denote bi 
and 62 as the basis vectors of the lattice, r = 62 /&i = 
u + iv characterizes the lattice type and Vf. = b^v is the 
area of a unit cell. The argument C of the theta function 
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is defined by scaling uj with 61, i.e. C, = w/bi. The 
explicit form of 0{C,t) is 



-, +00 



+ |)2+27r?C(n+^) 



(9) 



n— — 00 



In Ref.jg] it has been shown that from Eq.® one can 
obtain that 

= ^^5^exp(zi?. r)cxp(-rVa2), (10) 

if 

where Km-im2 — rniKi + 7712^^2 are the reciprocal lattice 
vectors, 



5k 



= (_l)mi+m2+mim2g-«c|i<'| /S^y'yj2 (11) 



and 



= (27r)2w~i[(umi)2 + (ma + umif 



(12) 



The condensate radius (r^) is modified to by the pres- 
ence of large number of vortices, and 



(13) 



Furthermore, with the help of Ea. (|10|l we can evaluate 
the self-interaction energy of each condensate 



dxdy\ip\'^ 



1 K.K' 



(14) 



In the larger vortex number limit, ■ko^ jvc ^ 1, we only 
keep the K = —K' terms in the numerator and K = Q 
term in the denominator. The integral (|14() can then be 
simplified to 



/ 



dxdy\Lp\ 
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27rcr2 
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K 



9K_ 
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(15) 



Then we can minimize the energy functional, includ- 
ing Eq.Q and Ea. lfT^ . to obtain the average vortex den- 
sity and the vortex lattice structure. We notice that all 
the information about the lattice type is contained in 
Ii . Minimizing Ii we will find the lattice structure being 
triangular. However, if some additional factors are in- 
cluded, such as the anisotropy of the confinement poten- 
tial, it is also possible for the lattice structure changing 
from triangular to others. [TTj [T ^ 



B. Double-Layer Triangular Vortex Lattice 

Now we begin to discuss two coupled rotating BECs 
by using QHMF theory. The inter-layer interaction and 
the coherent hopping between two condensates should be 



considered. Because in this model U12 -^U and the hop- 
ping terms are almost independent of the lattice structure 
as we will show later, we can assume that the inter-layer 
coupling will not change the lattice structure of each con- 
densate. In this subsection we will firstly focus on the 
triangular lattice. 

It has been found in Ref.*^ that 



dxdy\ipi\^\ip2\^ 



-y 



27rcr2 



K 



9k 
90 



cosA'-fo- (16) 



Because qk exponentially depend on \K\ as one can see 
from Ea. ljllll . only several terms such as goo, 9o.,±i, g±,o 
and 5±i,±i need to be considered. For triangular lattice 
we have Ii = 1.1596, and 



dxdy\ipi\'^\ip2\'^ 



1 



where 



f{n) ) = C ^ COS. K-ro 



(17) 



(18) 



with C = 0.0532. The minimum value of /(ro) is — 3C/2 
which occurs at rg = (61 -|-62)/3, and its maximum is 3C 
occurring at rg = 0. 

Using Eq.© we evaluate the coherence terms (for de- 
tails see the appendix ^ . Considering the number of 
vortices contained in the cloud — -kg^ jvc ^ 1 and 
following the same approximation made in Ref. 6], we 
drop all the if 7^ terms and obtain the simplified ex- 
pression of Eq. (|A10p is 



3? 



J dxdyipl(p2 = exp ^-N^^rlJ 



and 



dxdyiplip2 = 0. 



(19) 



(20) 



Similarly we obtain the real part of the two-particle co- 
herent hopping term as 



3? / dxdy{(plip2)'^ 



2TTa 



27rCT' 



-y 



K 



9K_ 

90 



exp ( -Ny—r^ 



■exp ( -iV„— 



(21) 



while its imaginary part vanishes. These coherent terms 
are exponentially dependent on the relative displacement 
rg, and the exponential factor Ny is very large for the case 
discussed in this paper. This indicates that the coherent 
terms will be exponentially small and the phase fluctu- 
ation will be enhanced when Tq is comparable to Vc- It 
means that if the competition results in a transition from 
To = to a non- vanishing rg, such a transition will be 
accompanied by the loss of phase coherence. 
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According to above discussions, the energy functional 
can be rewritten as following: 

E = --^cos^exp [-N^—rl] + h{ijj - f^)-^ 



-—-[2UI + Wi2+Wi2.f{ro) 



-2J7i2/cos26'exp ( -N^—rl )]. 



(22) 



In the following we will minimize the energy functional 
to obtain the mean field ground state, and then discuss 
the phase diagram. First of all we minimize E with re- 
spect to a. As U12/U is a small parameter, we can ex- 
pand the result to its first order, which yields that 



E 



t 

--COM 



AU12 



/(ro) + ycos20e- 



(23) 

Here A is defined as y^U{uj± — f2)/(7ra^). Two constant 
terms, 2A\fT\ and AU\2l {^flxV) have been neglected in 
the above expression. 

When t > 4 A\/I^U 126^^" ^""^ /U, the expectation value 
of the relative phase 6 tends to be zero, and then 



AUuVh 



2Ut 



8UH 



(24) 



While when t < AAy/TiUi2e ^" /U, 6 will gradually 
change from zero to 7r/2 with t decreasing, and 



tu 



cosf 



A^iAUi2 
the energy minimum will be 



exp 



E 



lQAUl2^/h 



AU12VT1 

2U 



2Vr 



(25) 



AU12 



firo). 
(26) 

From the ro dependence of the energy Ea. (|^ and 
Ea. (P5|l . one can easily find that the hopping energy favor 
a vanishing while /(rp) favors a non- vanishing tq. As 
schematically shown in Fig.® there possibly exist two 
local minima located at ro = and tq — (bi + b2)/3- 
The true ground state is determined by comparing the 
two local minima, denoted by Ei and E2 respectively. In 
the large t limit when the hopping energy is dominant, 
the ground state is at fo = as shown in the left side 
of Fig.Q. One may expect that Ei will be eventually 
larger than E2 as the right side one, and result in a first 
order transition of ro as t/Ui2 decreasing. 

At ro = (f'i+&2)/3 the coherent terms are small enough 
and can be neglected, the minimum value of E is there- 
fore independent of t and 6. One can easily find out from 
Eq.(|23|) and Eq.^ that 



E2 



3C AU12 



hU 



(27) 




^ r^ta''bunits)" 



FIG. 3: The minimum value of the energy functional i5 as a 
function of ro, Ea. l24ll and Ea. l26ll . has two possible shapes. 
In the large t limit it behaves like the upper one, and one 
may expect that it will eventually behave like the lower one 
as tjUvi decreasing. In the following text we will discuss 
whether and when such a transition will take place. 



We can also find from these two equations that E\ in- 
creases as t decreases, however there exists an upper 
bound, i.e. 



E, < 



AU12 



3C 



AU12 
2U 



(28) 



For the phase transition taking place, it is required that 
El > E2, i.e 



9C > h. 



(29) 



This condition can not be satisfied for the triangular vor- 
tex lattice, where h = 1.1596 and C = 0.0532. There- 
fore, the conclusion is that the two sets of vortex lattices 
will coincide throughout all of the coherent regime. 

As we have emphasized the underlying physics of the 
transition of ro is the competition between the coher- 
ent terms and the density-density interaction term. The 
coherent terms include one-particle hopping term (Pi^2, 
whose characteristic energy is t, and the two-particle hop- 
ping terms ipi'^ip2, whose characteristic energy is Ui2. 
When t ^ U12 the relative phase 9 between two layers 
will be zero, which is beneficial to one-particle hopping 
energy. We found that in this case the hopping energy 
will always dominate over the density-density interaction 
energy. With t decreasing, 6 will change from zero to 7r/2 
to minimize the two-particle hopping energy. Eventually 
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whether the transition can occur mostly depends on the 
competition between two-particle hopping term and the 
density-density interaction term. Since both these two 
terms share the same characteristic energy U12, which 
term will be dominant relates to the intrinsic proper- 
ties of the vortex lattice states. This can be seen from 
Ea. (|29|l . where / and C only depend on the lattice struc- 
ture. 

Hence if the lattice type is changed, the above conclu- 
sion must consequently be changed. We will generalize 
the above discussion to arbitrary lattice type and find out 
on what condition the transition can occur in the fourth 
subsection. 



C. Sliding Mode Frequency 

Before continuing our discussion on the transition, in 
this subsection we discuss briefly one new excitation 
mode, which is unique to this system. This mode is char- 
acterized by small amplitude oscillation of tq around its 
equilibrium position rg = 0. We call this mode sliding 
mode because the vortices in one layer oscillate in locked 
steps relative to the other layer, and for the optical lat- 
tice case (see the right side of Fig.(P)) this mode can 
be corresponding to Kelvin mode of vortex line in the 
conventional three-dimensional rotating BEC.[l3| 

We write down the propagate in the path integral rep- 
resentation as follows 



Z = d'i'* d'i' exp 



J dtd'^ri^*dt'i' - E{^*,^) 



To consider the oscillation of tq we have 



(30) 



(31) 



the second equality follows from the fact that only ip2 
contains tq . We notice that it is necessary to go beyond 
the mean field approximation and take the density fluc- 
tuation into account in order to obtain a dynamic term 
of tq. The fluctuation of n2 denoted by dn contains two 
contributions. One comes from the inter-layer coupling, 
which is independent of spatial coordinates. The cou- 
pling to this part can not induce a dynamic term of tq 
because J d'^r(p* drg^p vanishes at ro = 0. Therefore in 
order to obtain a non- vanishing dynamic term we should 
include the intra-layer local density fluctuation bringing 
the condensate out of the lowest Landau levels. The dy- 
namic term in Ea. (|30|l is then written as 



dtSro / d'^rSn{r)i(p2dr„ip2- 



(32) 



We can then expand the energy functional around the 
saddle point to the second order, the result is 



cos 9N„ — 5rn 



U 



„/i cos26'7V„--^(5rn 

^ 2Vr 



d^r4U\ip2\'^dn{rf 



(33) 



The term /(ro) has been neglected in the above equation 
because the coefhcient of second order expansion in the 
neighborhood of ro = is relative small. Here cto is the 
saddle point of a. We first integrate the field Sn{r) out 
to obtain a dynamic term of ro, the effective Lagrangian 
describing the oscillation of ro then reads 



L 



2 ^"^0 



2Vr 



t COS 9 



Uh cos 261 



Sri (34) 



Here mcfr is the effective dynamic mass of the collectively 
oscillation motion defined by 



d^r- 



-{ip*2dr„ip2)'^ 



SU\ip2\^ 

Hence the oscillation frequency is 



(35) 



tcost 



Uh COS 29 



• (36) 

rries 



What should be emphasized here is that this frequency 
is a smooth function of cos^. As shown in the above 
section 9 will gradually change from zero to 7r/2 below 
the critical point t/Ui2 = AA^/h/U. This leads to the 
key prediction of this subsection that the sliding mode 
frequency will consequently exhibit a second order dis- 
continuity at the critical point. 



D. General Lattice Type 

To discuss a vortex lattice with arbitrary lattice struc- 
ture, instead of Ea. H18|l we define 



COS K ■ fo, 



(37) 



and denote its minimum by M, the condition ()29|l for a 
transition in ro is then modified to 



h-2-2M > 0. 



(38) 



For flat lattice gKi _i is much larger than 1 and gKi^o ■ 
As an example for the lattice with argr = 7r/6, 





2 




90 




90 



0.1857, 



(39) 



it is much larger than \gKo,i/9o\'^ and |gifi,o/5oP, which 
are both equal to 0.0019. In this case the minimum of 
/(ro) is approximately 



M = -/(ro = 0) = l-/i, 



(40) 



which can be reached when ro = (fei + 62)/4. 

Hence the condition Ea. H38l) can be satisfied when the 
lattice is flat enough that satisfies argr < 0.6728. The 
transition takes place when Ei = E2, i.e. 



(41) 
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Here A/ 17 is a dimensionless parameter and can be tuned 
in a wide range. As an example, we show in Fig.Q) the 
transition line for a flat lattice with argr = 7r/6. In the 
upper side the vortices in two layers coincide with each 
other and the vortex lines are parallel to the z axis. On 
the other side two sets of lattices are staggered, and the 
phase coherence between layers is lost. 

Therefore to experimentally observe such a transition 
in QHMF regime, the first step is to produce a sufficient 
flat vortex lattice. Although so far most observed vor- 
tex lattices are triangular, it is still possible to achieve 
other lattice structure by using some special methods. 
Recently in Ref.[l3] the author shown within the LLL 
approximation that for a condensate with small number 
of vortices the lattice structure can be compressed to be 
quite flat due to the anisotropy of confinement potential. 




FIG. 4: The phase boundary between ro = coherence 
regime and ro 7^ Fock regime, for different values of AjU . 
In the regime above the line r = and below the line rb 7^ 0, 
there is a first order transition when crossing the line. 




our results as follows and the phase diagram is shown in 
Fig.®. 

1. In the coherent regime, for triangular lattice a transi- 
tion of fo is absent and tq will always remain zero when 
the coherent hopping terms depending on the relative 
phase between two layers are introduced into the energy 
functional. When t decreases the relative phase between 
two condensates will experience a second order change. 
The change of phase manifests itself in a second order 
change of the sliding mode frequency. The phase dia- 
gram is shown in the left side of Fig. © 

2. When the lattice is compressed to be quite flat, a 
transition of ro will occur when Ui2/t exceeds some crit- 
ical value. The number fluctuation will be suppressed 
after the jumping of tq and a flrst order transition from 
the coherent regime to the Fock regime will be induced. 
It is noticed that such a transition from coherent regime 
to Fock regime is different from that discussed in the 
double- well BEG case, because it is driven by t/Ui2 in- 
stead of t/U , and it is a first order transition instead of 
crossover. The phase diagram is shown in the right-side 
of Fig.® 
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FIG. 5: Left: The unit cell. Right: The schematic exhibition 
of triangular lattice and flat lattice. 



FIG. 6: The phase diagram for the double-layer vortex lattice. 
The upper is for triangular lattice and the lower is for flat 
lattice 



E. Summarizes 

In this section we have discussed the phase diagram 
of double-layer vortex lattice state, when both rotating 
condensates are in the QHMF regime. We summarize 



IV. THOMAS-FERMI REGIME 

We have mentioned that there are two different mean 
field descriptions of rotating BEG. In the last section we 
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FIG. 7: The phase diagram for the double-layer vortex lattice. 
This phase diagram is obtain when each rotating BEG is in 
Thomas- Fermi regime and the condition Eg. 15811 is satisfied. 



have focused on the QHMF regime. Before investigat- 
ing the same issue when both condensates are in TFMF 
regime, we would like to have a brief discussion on the 
difference between two regimes. 

As shown in the last section, QHMF works when the 
kinetic energy is much larger than the interaction en- 
ergy. The wave functions are eigenstates of single par- 
ticle Hamiltonian, and the mean value of h^y only de- 
pends on the average vortex density, and {h^^y) neither 
depends on the winding number of each vortex nor the 
structure of vortex lattice. That the singly quantized vor- 
tices are arranged into triangular lattice is determined by 
taking the self-interaction energy into account. In con- 
trast, TFMF is obtained by neglecting the kinetic energy, 
and the ground state density distribution is the result of 
the balance between effective trapping energy and inter- 
action energy. The density distribution is assumed to be 
not remarkably changed and keeps a form of an inverted 
parabola when large number of vortices are contained. In 
this regime the interaction energy is independent of both 
the vortex winding number and the lattice structure. The 
kinetic energy caused by superfluid currents around the 
centers of vortices will induce an effective repulsive inter- 
action between them. As a result the vortices are singly 
quantized and arrange into triangular lattice. 

Besides there are also phenomenological differences be- 
tween two regimes. In the TFMF regime the density pro- 
file is an inverted parabola and there exists a vortex core 
structure for each vortex. A vortex core structure means 
that there is a characteristic length ^, inside which the su- 
perfluid density drops to zero rapidly, and outside which 
the superfluid density recovers the value free from vor- 
tices. Because the healing length increases as interaction 
strength decreases, when entering the QHMF regime ^ is 
comparable to the inter- vortex spacing and no longer well 
defined. The individual vortex core structure disappears. 

Therefore to deal with the TFMF regime we should 
use another wavefunction ansatz, which will be quite dif- 
ferent from the one in QHMF regime. We know that the 



Thomas-Fermi density profile is 
1 / 1 



2g 



m(LU 



(42) 



In the presence of vortex lattice we assume that the den- 
sity profile is modified in the following way 



p(x po 



(43) 



where is the centers of vortices. ^ is the vortex core 
side. Here we assume the ^ is the same for all vortices. 
This ansatz is believed to be valid in the regime tt^'^ < 
Vc ^ ttR^, where R is the Thomas-Fermi radius of the 
condensate. 
We notice that 



Pq 



nj) cxp 



(44) 



in the large vortices limits, we can assume the area of 
a unit cell is much smaller than the size of condensate, 
approximately we have 



Po{rij)vc 



d rpa{r) = 1. 



(45) 



Hence Ea. H44|l reduces to 1— tt^^/wc- Such approximation 
will be often used in the following derivation, the spirit 
of which is essentially keeping accuracy to the first order 
of TT^"^ /vc and neglectirig Vr/ {ttR'^). Similar approach has 
also been used in Ref.jl^l in the study of single compo- 
nent vortex lattice state. 

The density distribution of vortex lattice state is 



1 



P = 



1 



Pa 



E 



Poinj) exp 



(r-nj)' 



(46) 



(47) 



and the wave function is written as 

Here f{z) = IK-^ ~ ^'^d can be described by the 

Jacobi theta function mentioned in the previous section. 

In this regime, we can neglect the overlapping between 
vortex cores and then have 

\Mr)\'\Mr)\' - ^ 



2{P0 



+p(r)po(r., +ro)e-('-'---''«)'/«' 



(48) 
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Approximately we can assume poifij + tq) ~ Poifij) be- 
cause ro is less than one lattice spacing and much smaller 
than the condensate size. Hence in Ea. H48|l only the last 
term involves rp, explicitly 



/ 



rij 



(49) 



where I2 is defined as / p^iPr. And therefore 

1 - 2^ + #e"^«^ 



dxdy\tpi\ \lp2\ 



-h. (50) 



We notice that the larger rg, the smaller the density- 
density interaction energy. Hence its minimum is reached 
when ro = (&i -I- b2)/2. 

Now we begin to discuss the coherent hopping terms 
in TF regime where 



'^qhl^qh2 
'?qhl<?qh2 



(51) 



using the result of Eq. ljA9p and taking the approximation 
of dropping all the terms with K 7^ 0, we obtain that 



3iiplip2 = cos ( — ro ) , (52) 



and 



S(Pi(p2 = sin ( — r X fo 



(53) 



From a careful analysis in the appendix!^ we conclude 
that at ro=0 



3? J dxdyipl(p2 — 1, 



(54) 



and 



r 1 _ ^lil 
^ / dxdyipl^ipl = I2- ^r-r = ^3- (55) 

Their imaginary parts vanish due to the spatial reflection 
symmetry. These coherent terms decrease as tq increases. 
When To reaches its maximum value (61 -I- 62)72 these 
terms are small enough that can be neglected. 

In the following we will find the ground state by com- 
paring the minimum values of the energy function at 
ro = and ro = (61 + 62)72. We notice that at ro = 

E = -^cose^ 4C/12/3 + 2J7i2 cos 26I/3 (56) 



and at ro = (61 -|- 62)72, 



E = AUuh 



1-2^ 





(57) 



Following the similar procedure performed in the previ- 
ous section, we find that when 



> 0.4 



(58) 



a transition of ro will occur as t decreases. When the 
condition (|58|l is satisfied, the critical value oft/Uu is 



t 

Th2 



8h 



\ 



■ire 



1 _ 



(59) 



When t/Ui2 adiabatically decreases below this critical 
value, ro will jump from zero to a non-zero value, and 
the inter-layer quantum tunnelling will be suppressed im- 
mediately. The phase diagram is shown in Fig(|Zj). It is 
physically similar to the right side of Fig. 10, but illus- 
trated in an alternative way. 



V. DISCUSSIONS AND CONCLUSIONS 

So far we have studied the phase diagram of double- 
layer rotating BEC in quantum Hall limit and Thomas- 
Fermi limit respectively. The complexity as illustrated 
in above discussions once again demonstrates the rich 
physics of rotating BEC and quantum coherence effect. 
Our model contains rich physics due to the four compet- 
ing energies. The competition between intra-layer kinetic 
energy, whose characteristic value is hfl, and the intra- 
layer interaction energy U determines the properties of 
vortex lattice. The competition between inter-layer ki- 
netic energy, whose characteristic value is t, and U de- 
termines the phase fiuctuation between layers. The effect 
of the competition between t and U12 is emphasized in 
this paper. However these three effects are entangled to- 
gether and interact strongly with each other, that leads 
to the abundant physical phenomena. 

What is most interesting is that under certain condi- 
tion the system will exhibit a new kind of quantum phase 
transition as t/Ui2 decreases. The transition is charac- 
terized by two sets of vortex lattice being staggered, and 
consequently the loss of phase coherence after the transi- 
tion. Hence it presents a novel mechanism for superfluid 
to Mott transition. Furthermore, the condition that such 
a transition can happen depends on the intrinsic prop- 
erties of each condensate. In TFMF regime, the condi- 
tion is that the ratio of vortex core area to the area of 
unit cell should be larger than a critical value. While in 
QHMF regime the condition is that the lattice structure 
should be flat enough. We remark that the different cri- 
teria reflect the essential difference of these two physical 
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regimes. Because how much energy the density-density 
interaction term can gain in the staggered lattices phase 
depends on the density undulation caused by vortices, 
in the TFMF regime the density undulation is mostly 
caused by vortex core structure. While in QHMF regime 
individual vortices cores are smeared, the density and 
phase singularities are strongly coupled, the density un- 
dulation therefore directly relies on how vortices arrange 
themselves. This indicates that the study of inter-layer 
coupling can be used as a powerful tool to reveal the 
intra-layer physics. 

There are still many opening interesting questions. So 
far we still have no effective method to study the in- 
termediate region between TFMF regime and QHMF 
regime, which may be relevant to the recent experiments 
of fast rotating BEC. 16] Besides, if the rotating BEC is 
loaded into a one-dimensional optical trap, it is possible 
to achieve the regime that the atom number in each site is 
comparable to the vortex number, and in each layer the 
Bose atoms are strongly correlated. [l7j Such a system 
may behave like a bosonic multi-layer fractional quan- 
tum Hall system. The discussion made in this model can 
also be applied to a rotating spin- 1/2 condensate, with 
two hyperfine spin states coupled by photons. However 
in that case U12 may be comparable with U, and the sit- 
uation will be more complicated. More fruitful physics 
is expected in further investigation, and we hope that 
the theoretical results obtained in the present work will 
stimulate more experiments. 
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APPENDIX A: CALCULATION OF COHERENT 
HOPPING TERMS IN QHMF REGIME 

In this appendix we will calculate the coherent hopping 
terms between two layers with the help of the Jacobi 
theta function in the QHMF regime. Using Eq.© we 
have 



^(C,T)0(C+fo,T) 



E 

m — — 00 



+CXD 

X E (-1) 

712 — — 00 



n2gi7rT(n2 + |)^+27r'tC(n2 + |) 



^(-l)rexp[2 



Here is explicitly written as 



+00 

E ' 

m — — 00 



m2 

E( 



^ TT m ' ^ + TT i ( a: + i y ) ' g 2 -TT i m 2 n 1 



-1, 

1 



i(mi+l)m2 



11 — 2Tzy+i7ZXQ — TzyQ-iriTTm2)'ni' — 



{'mi + l)m2 ^ — Tr{m2+XQ+umi+2iy+iya) /2v 



By defining 



K = 



2'Kmix — 2tt 



1712 + umi 



V 



1 



and 



ro = xqx + yoy, 



we can verify that 
Traiomi 



T^yo{m2 + umi)/v ^ -tq- K 



and 



- nyomi - 7ra;o(m2 + umi)/v = -ro x K. 
Using Ea. ljA2(l . (|A5(l and ljA6|l we can obtain that 

r(c,T)0(C-Hfo,T) 

= E/^e''^"^""'" 

K 



(A2) 
(A3) 
(A4) 
(A5) 
(A6) 

,(A7) 



where 



(-1) 



mi+m2+mim2 



/2v 



^^[(roy.K)+i(ro-K)]-v^\K\^ /(Hit) ^ 



(A8) 



Therefore 



iplif2 OC ^ 
K 



iK-r 



exp 



X exp 



' — {2iy + 2:0 + iyoY 
2v 

r'^ + {r + ro)^ 
2^ 



E/^ 



iK-r 



K 

X exp 



and 



mmix — —VTTmf + itt^Xq + «yo)'^i]-^mi i 

(Al) 

where fg = ro/bi, mi = n2 — ni and m' = ui + n2 + 1. 



KK' 



r r i(K + K')-f 



—i{fx fo) 

Vr 



(A9) 



X exp 



2 



— i[r X ro) 

Vc 
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Then we can perform a integral over the two- 
dimensional space. After considering the normalized con- 
ditions, it yields 



0.7% 



J dxdyip\ip2 = 



K 



(AlO) 



APPENDIX B: CALCULATION OF COHERENT 
HOPPING TERMS IN THE TFMF REGIME 

In this appendix we will calculate the energy of co- 
herent hopping term in the TFMF regime. Begin with 
Ea. (|5^ we have 

J (fripltp2 = ^ J (fry/pip2 cos[-^(fx fo)] = 

unit cell 

^ / (fry/pip2 cos[— (f- r7j) X fo]cos[—{fij x tq)], 

''J u.c 

(Bl) 

the sin term is cancelled out due to the spatial reflection 
symmetry. 

Approximately in each unit cell we take 



Pi 



1 _ nil 



[I — exp 



and 
hence 



- [1 — exp 



(B2) 



(B3) 



(f'r^piP2 cos[— (f- rj, ) X fo] 



Po(ri, 



(fr'\l{l~e ^)(l-e *' ) cos[— r' x ro] 



Vc I d^r'\j (1 - e - e ^ ^)cos[7rr' x fo]. 

(B4) 

Here r' is defined as r — . The second equality follows 
from a scaling that r' and rg are replaced by r'/y^ and 
''0 / We notice that for the determined lattice struc- 
ture the integral in Ea. (|B4() only depends on Vc/^"^ and 
Tq, which is denoted as F and can be obtained numeri- 
cally. 




0.2 0.4^ 0.6 0.8 1 



FIG. 8: Upper: The integral F as a function of ■^§=. Lower: 
The integral / as a function of -^=- Vc is set to unity, T = 
■K^^/vc is 0.3 and is 225. 



Therefore 

Eq.dBll) = — il^^j2po{nj)vccos[—{nj X ro)] 

1 — — ^ Vc 

^ ^ — / drrdd pQ{r) cos[ — rrosin^] 



1 



= 27r— J drrpo{r)Jo{^rrQ) 



(B5) 



where Jq is Bessel function of zero order. At tq = 0, 
equals to 1 — tt^^/wc. As tq increases F will decrease. 
The numerical result is shown in Fig.© 

Besides, let t = r/R with R being the Thomas-Fermi 
radius we have 



27r / drrpo{r)Jo{—rro) 



—m(Lj^ - n^)R'^ I dtUl - t^)Jo ( —roRt 
2.9 J \Vc 



8 f7rroR\ 
{uroRY/vl ^\ Vc ) 



(B6) 



The second equality follows from the Sonine integral 
identity and the fact that po is normalized. It yields 
unit at tq = and therefore Ea. (jBl|) results in unit as 
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we expected. Notice that 




where Ny is the number of vortices contained in the 
cloud, and its typical value is 100 ~ 250 in current ex- 
periments. We choose ^/Ny being 15 in the following 
analysis, the result of Ea. (|B6l) is therefore 

Denoting the righthand side of Ea. (|B8|) by /, / as a func- 
tion of ro/y/v^ is plotted in the righthand side of Fig.||SJ). 
We find that / is of the order lO"** and the hopping en- 
ergy can be neglected when tq = (&i -I- b2)/2. 

In the same way we can calculate the two-particle hop- 



ping energy, 

where W function is defined as the integral 
J dV(l-e"^)(l-e"*"""°' * )cos(27rr X ro) . (BIO) 

u.c 

Similarly when tq = {bi + b2)/2 Ea. ljB9p is about of the 
order 10~® and need not be considered. 
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